Knowledge Distillation with Contrastive Inter-Class Relationship

Topic Distillation with Knowledge Agents

This is the second year that our group participates in TREC’s Web track. Our experiments focused on the Topic distillation task. Our main goal was to experiment with the Knowledge Agent (KA) technology [1], previously developed at our Lab, for this particular task. The knowledge agent approach was designed to enhance Web search results by utilizing domain knowledge. We first describe the generi...

9. Contrastive Knowledge

Contrastive Knowledge Surveyed

Suppose that Ann says, “Keith knows that the bank will be open tomorrow.” Her audience may well agree. Her knowledge ascription may seem true. But now suppose that Ben—in a different context—also says “Keith knows that the bank will be open tomorrow.” His audience may well disagree. His knowledge ascription may seem false. Indeed, a number of philosophers have claimed that people’s intuitions a...

Sequence-Level Knowledge Distillation

Neural machine translation (NMT) offers a novel alternative formulation of translation that is potentially simpler than statistical approaches. However to reach competitive performance, NMT models need to be exceedingly large. In this paper we consider applying knowledge distillation approaches (Bucila et al., 2006; Hinton et al., 2015) that have proven successful for reducing the size of neura...

Supplementary Materials of Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation

Spatial features have been demonstrated to be helpful in visual tasks such as object detection, image retrieval, and semantic segmentation [13, 12, 9, 6, 1, 2, 7]. For visual relationship detection task, spatial features such as the relative location and size of two objects are informative for predicate prediction. For example, relative location is a discriminative feature for predicates “under...